Moving image encoding apparatus

ABSTRACT

In a moving image encoding apparatus of this invention, a generated bit amount extracting section ( 21 ) counts generated bits of variable-length-encoded data in units of macroblocks, and a reference generated bit count is set in a parameter value control section ( 22 ) as an update condition for a parameter value in each block. The boundary value of the parameter value is set in an intra/inter determination section ( 23 ) as a criterion for selection of the intra-encoding mode. In the inter-frame encoding mode, the parameter value control section ( 22 ) updates the parameter value of a block in which the generated bit count value of encoded data obtained by the generated bit amount extracting section ( 21 ) exceeds a reference value. The intra/inter determination section ( 23 ) determines that the influence of a data error is large in a block in which the parameter value is continuously updated and exceeds the boundary value, and switches the current mode to the intra-encoding mode to perform refresh operation.

TECHNICAL FIELD

[0001] The present invention relates to a moving image encodingapparatus for compressing and encoding a video signal (to be referred toas a moving image signal hereinafter) of a moving image with highefficiency.

BACKGROUND ART

[0002] A conventional moving image encoding apparatus is designed tocompress and encode a moving image signal with high efficiency by usingthe statistical characteristics of a moving image signal, i.e.,correlations between intraframe pixels and between interframe pixels,thereby realizing a great reduction in information amount.

[0003] As basic schemes of compressing and encoding moving imagesignals, a predictive encoding scheme and transformational encodingscheme are available. The predictive encoding scheme uses correlationsin time domains. The transformational encoding scheme uses correlationsin spatial domains. As a predictive encoding scheme, a motioncompensation predictive scheme of encoding a difference signal betweenan original image and a predicted image obtained by motion compensationprediction is known. As a transformation encoding scheme, an orthogonaltransformation scheme is known, in which a frame is divided into blockseach constituting of a plurality of pixels, the respective pixelcomponents are orthogonally transformed into components in frequencydomains in units of blocks by DCT (Discrete Cosine Transform), and theorthogonal transformation coefficients (DCT coefficients) of theresultant frequency components are quantized, therebyvariable-length-encoding the quantized coefficients.

[0004] In general, a moving image encoding scheme as a combination ofthe above motion compensation prediction scheme and orthogonaltransformation scheme is used. As such a scheme, a scheme called H261and H263 has been standardized by ITU-T (International TelecommunicationUnion-Telecommunication Standardization Sector), and a scheme calledMPEG (Moving Picture Experts Group) has been standardized by the imagecompression standardization group established under ISO (InternationalOrganization for Standardization). In this MPEG scheme, encoding isdefined to be performed in units of blocks called macroblocks, eachconsisting of 16×16 pixels.

[0005] A general moving image encoding apparatus for encoding movingimage signals by using a combination of the motion compensationprediction scheme and orthogonal transformation scheme in the abovemanner includes an intra-encoding scheme and inter-encoding scheme asencoding modes. In the intra-encoding mode, the respective pixelcomponents in input frames are encoded in units of macroblocks withoutmotion compensation prediction. In the inter-encoding scheme, differencevalues between a motion compensation predicted image and an originalimage are obtained, and the resultant data are encoded in units ofmacroblocks.

[0006] In this case, the encoding apparatus in the interframe encodingmode generates decoded image data by internally decoding the dataobtained by encoding a given frame A. This data is referred to as localdecoded image data and stored in the frame memory. In encoding anotherframe B, the encoding apparatus performs motion estimation by using boththe local decoded image data stored in the frame memory and the inputimage data, and generates a motion compensation predicted image on thebasis of the estimation result. Difference values between this motioncompensation predicted image and the original image are obtained, andthe difference data are encoded and output to the transmission path.

[0007] The moving image decoding apparatus on the decoding side(destination) stores the decoded image data of the frame A in the framememory as in the encoding side. Upon reception of the encoded differencedata, first of all, the decoding apparatus generates a motioncompensation predicted image from the decoded data of the frame A. Ifthere is no error in the received data of the frame A, since the decodedimage of the frame A matches the local decoded image of the frame A onthe encoding apparatus side, as a matter of course, the motioncompensation predicted image generated from this decoded data matchesthe motion compensation predicted image on the encoding apparatus side.

[0008] Since the encoding apparatus sends out the difference imagebetween the original image and the motion compensation predicted image,the decoding apparatus can generate a decoded image of the frame B byadding the motion compensation predicted image to the receiveddifference image.

[0009] The encoding apparatus generates a motion compensation predictedimage by cutting data nearest to the pixel value of an input block fromlocal decoded image data. If, therefore, the data of the frame Areceived by the decoding apparatus contains an error, a correct decodedimage of the frame A cannot be generated. As a consequence, all imagescut and generated from the error-containing portion of the motioncompensation predicted image become erroneous data. If, therefore, theerror-containing portion is cut as a plurality of motion compensationpredicted image blocks, the error is spatially diffused. These erroneousdata remain until refresh processing is performed by intra-encoding.

[0010] Same motion compensation prediction processing is performed forthe next frame. As a consequence, on the decoding apparatus side,decoded images generated by using motion compensation predicted imagesgenerated from the erroneous data are stored in the frame memory andused to generate a motion compensation predicted image of the nextframe. As such processing is repeated, the influences of erroneous datasequentially propagate. These erroneous data remain until refreshprocessing is performed by intra-encoding.

[0011] For the reasons described above, on the moving image encodingapparatus side, refresh must be periodically performed by intra-encodingto replace all the destroyed data with new image data, thus preventingthe propagation of a data error. In addition, the influences oftransmission encoding errors on the receiving side must be minimized.

[0012] With regard to refresh by intra-encoding, for example, accordingto ITU-T H.263, as the first scheme, a technique of performing refreshat least once while one macroblock is encoded 132 times is defined. Tomeet this definition, a technique of suppressing the propagation of adata error by cyclically refreshing a predetermined macroblock byintra-encoding is generally used.

[0013] In addition, according to MPEG-4, as the second scheme, atechnique of performing refresh by aperiodically intra-encoding onemacroblock is defined. In this technique, difference values between apredicted image and a current image are obtained in units ofmacroblocks, and macroblocks corresponding to difference values equal toor larger than a reference value are extracted as important areas thatrequire refresh. Refresh is then sequentially performed for themacroblocks in the important areas in descending order of priority.

DISCLOSURE OF INVENTION

[0014] The above conventional refresh method has the following problems.

[0015] First of all, as in the first scheme defined in ITU-T H.263described above, the following problem arises in the method ofcyclically performing refresh. In this method, for a macroblock forwhich intra-encoding has been performed in a given frame, refresh is notperformed until the next intra-encoding period. For this reason, if thedata of a given macroblock is destroyed immediately after refresh isperformed once by intra-encoding in one frame, a deterioration in imagequality remains until the next refresh is performed, and the data erroris diffused to neighboring macroblocks by motion compensation predictionprocessing.

[0016] As in the second scheme defined in MPEG-4 described above, thefollowing problem arises in the method of performing refresh in units ofmacroblocks by aperiodical intra-encoding. In this method, if the numberof macroblocks contained in an important area recognized as a refreshtarget in one frame is larger than the total number of macroblocks thatcan be actually refreshed in one frame, all the macroblocks recognizedas refresh targets cannot be refreshed within one frame. In this case,even if it is determined that a given macroblock needs to be refreshed,when the priority of this macroblock is low, the macroblock is notrefreshed. For this reason, if new macroblocks are extracted as refreshtargets in the next frame, the number of macroblocks as refresh targets,including macroblocks that remain unrefreshed, increases. As aconsequence, a residual deterioration in image quality graduallyincreases.

[0017] In addition, whether a given macroblock is a refresh target ornot is determined by only the pixel difference value in one frame.Assume that this pixel difference value is large. Even in this case, ifthe number of DCT coefficients is small upon discrete cosine transform,the amount of encoded data used for one macroblock decreases, and theprobability of mixing of an error decreases. Therefore, correctdetermination cannot be made by the technique of performingdetermination on the basis of only the pixel difference value in oneframe.

[0018] The present invention has been made to solve these problems, andhas as its object to provide an moving image encoding apparatus whichcan accurately determine the necessity of refresh by intra-encoding inthe inter-frame encoding mode, and prevent from diffusing a data error,even if it occurs, by motion compensation prediction processing.

[0019] It is an object of the present invention, in particular, toprevent a deterioration in image quality from remaining and a data errorfrom diffusing when refresh is cyclically performed. It is anotherobject of the present invention to prevent unrefreshed macroblocks fromremaining and remaining blocks from increasing in number whenaperiodical refresh is performed in units of macroblocks. It is stillanother object of the present invention to properly determine whether amacroblock in one frame is a refresh target or not.

[0020] In order to achieve the above objects, a moving image encodingapparatus according to the present invention is configured in thefollowing manner.

[0021] (1) A moving image encoding apparatus including

[0022] block generating means (12) for receiving moving image data inunits of frames, dividing the received data into a plurality of blocks,and outputting the data of the respective blocks in a predeterminedorder, and

[0023] mode-specific encoding means (11, 13-20), having intra-encodingmode and inter-frame encoding mode, for encoding in a designated modethe data of the respective blocks sequentially output from the blockgenerating means, is characterized by comprising:

[0024] information extraction means (21, 24) for extracting informationfor determination of a degree of influence of a data error from anencoded block processed by the mode-specific encoding means (11, 13-20);

[0025] parameter value control means (22) for, when intra-encoding isperformed by the mode-specific encoding means (11, 13-20), setting aninitial value as a parameter value of a corresponding block in advance,comparing the information extracted by the information extraction means(21, 24) with a predetermined reference value, and updating and storingthe parameter value on the basis of the comparison result; and

[0026] intra/inter determination means (23) for, when data of a block ofa next frame, output from the block generating means (11), is to beencoded, comparing a parameter value of a corresponding block, stored inthe parameter value control means (22), with a predetermined prescribedvalue, determining on the basis of the comparison result whether toexecute refresh by intra-encoding, and designating a mode based on thedetermination result with respect to the mode-specific encoding means(11, 13-20).

[0027] (2) The moving image encoding apparatus of (1) is characterizedin that the mode-specific encoding means (11, 13-20) encodes pixelcomponents in an input frame in units of blocks without performingmotion compensation prediction in the intra-encoding mode, and obtainingdifference values between an original image and a motion compensationpredicted image obtained by motion compensation prediction and encodingthe difference values in units of blocks in the inter-frame encodingmode.

[0028] (3) The moving image encoding apparatus of (1) is characterizedin that the information extraction means (21) extracts a generatedencoded data amount from an encoded output from the mode-specificencoding means as information for determination of the degree ofinfluence of the data error.

[0029] (4) The moving image encoding apparatus of (1) is characterizedin that when the mode-specific encoding means (11, 13-20) performsdiscrete cosine transform processing for block data, the informationextraction means (24) extracts a total number of non-zero coefficientsof coefficients obtained by the discrete cosine transform processing asinformation for determination of the degree of influence of the dataerror.

[0030] In the moving image encoding apparatus having the arrangements of(1) to (4), the information extraction means (21, 24) extractsinformation (the generated encoded data amount in (2) and the totalnon-zero coefficient count in (3)) for the determination of the degreeof influence of a data error from each encoded block, a reference valuefor extracted information is set in the parameter value control means(22) as an update condition for a parameter value in each block, and aprescribed parameter value is set in the intra/inter determination means(23) as a criterion for the selection of the intra-encoding mode. In theinter-frame encoding mode, for a block in which the extractedinformation obtained by the information extraction means (21, 24) islarger than the reference value, the parameter value control means (22)updates the parameter value of the block. The intra/inter determinationmeans (23) determines that a data error is likely to occur and have aconsiderable influence in a block in which the parameter value iscontinuously updated and becomes larger than a boundary value, andswitches the current mode to the intra-encoding mode to perform refreshoperation.

[0031] With this processing, whether refresh is required for eachmacroblock in one frame can be properly determined. Even if thetechnique of cyclically performing refresh by intra-encoding is used, amacroblock that requires refresh is refreshed when it is required uponswitching to the intra-encoding mode without waiting for the nextrefresh cycle. This makes it possible to prevent a deterioration inimage quality due to a data error from remaining and a data error fromdiffusing.

[0032] (5) The moving image encoding apparatus of (1) is characterizedby further comprising reference value/prescribed value changing means(25) for detecting an error rate in a transmission path, and changing atleast one of a reference value held in the parameter value control meansand a prescribed value held in the intra/inter determination means (23)on the basis of the detected error rate.

[0033] According to the arrangement of (5), an error rate in atransmission path is detected, and both or one of the update referencevalue for updating the parameter value and the criterion value for theexecution of the intra-encoding mode is changed in accordance with thedetected error rate, thereby setting a condition for the execution ofrefresh operation in the intra-encoding mode in accordance with theerror rate in the transmission path. This can prevent a deterioration inimage quality from remaining on the moving image decoding apparatusside.

[0034] (6) The moving image encoding apparatus of (1) is characterizedby further comprising encoded data storage means (26) for storingencoded data output from the mode-specific encoding means (11, 13-20) inunits of blocks, and reference value/prescribed value changing means(27) for obtaining an encoded data amount stored in the means (26), andchanging at least one of a reference value held in the parameter valuecontrol means and a prescribed value held in the intra/interdetermination means (23) on the basis of the encoded data amount.

[0035] According to the arrangement of (6), both or one of the updatereference value for updating the parameter value and the criterion valuefor the execution of intra-encoding is changed by using the encoded datastorage means (26) for keeping the transmission rate constant inaccordance with the amount of encoded data stored in the encoded datastorage means (26), thereby setting a condition for the execution ofrefresh operation in the intra-encoding mode in accordance with theamount of encoded transmission data. This can prevent a deterioration-inimage quality from remaining on the moving image decoding apparatusside.

[0036] (7) The moving image encoding apparatus of (1) is characterizedin that the parameter value control means (22) adds one to the parametervalue when information extracted by the information extraction means islarger than a reference value, and adds 0 to the parameter valueotherwise.

[0037] According to the arrangement of (7), it can be determined foreach block from a parameter value that the generated encoded amount ortotal zero-coefficient count becomes large upon continuous execution ofinter-encoding.

BRIEF DESCRIPTION OF DRAWINGS

[0038]FIG. 1 is a block diagram showing the arrangement of a movingimage encoding apparatus according to the first embodiment of thepresent invention;

[0039]FIG. 2 is a flow chart showing a procedure for encoding in themoving image encoding apparatus according to the first to fourthembodiments;

[0040]FIG. 3 is a flow chart showing a procedure for executing refreshby intra-encoding in the first embodiment;

[0041]FIG. 4 is a block diagram showing the arrangement of a movingimage encoding apparatus according to the second embodiment of thepresent invention;

[0042]FIG. 5 is a flow chart showing a procedure for executing refreshby intra-encoding in the second embodiment;

[0043]FIG. 6 is a block diagram showing the arrangement of a movingimage encoding apparatus according to the third embodiment of thepresent invention;

[0044]FIG. 7 is a flow chart showing a procedure for executing refreshby intra-encoding in the third embodiment;

[0045]FIG. 8 is a block diagram showing the arrangement of a movingimage encoding apparatus according to the fourth embodiment of thepresent invention; and

[0046]FIG. 9 is a flow chart showing a procedure for executing refreshby intra-encoding in the fourth embodiment.

BEST MODE FOR CARRYING OUT OF THE INVENTION

[0047] Embodiments of the present invention will be described in detailbelow with reference to the views of the accompanying drawing.

[0048] (First Embodiment)

[0049]FIG. 1 is a block diagram showing the arrangement of a movingimage encoding apparatus according to the first embodiment of thepresent invention.

[0050] A moving image encoding apparatus according to the firstembodiment is characterized in that the number of encoded bits generatedby variable-length-encoding data obtained by DCT coefficientquantization are monitored in units of macroblocks, thereby choosingbetween intra-encoding and inter-encoding for macroblocks to beprocessed in the next frame. More specifically, if the amount of encodeddata generated by variable length encoding is large, it is determinedthat a large variation is occurring in the input image. In accordancewith this determination, the number of bits of encoded data is counted.More specifically, if the amount of encoded data generated by variablelength encoding is large, it is determined that a large variation isoccurring in the input image. In accordance with this determination, thenumber of bits of encoded data is counted. When the number of times thiscount value becomes larger than a reference value exceeds apredetermined value, refreshing operation is performed byintra-encoding.

[0051] The moving image encoding apparatus shown in FIG. 1 is comprisedof a mode switching section 11, block generating section 12, subtracter13, DCT (Discrete Cosine Transform)-quantizing section 14,variable-length encoding section 15, frame memory 16, motion estimationsection 17, motion compensation section 18, IDCT (Inverse DiscreteCosine Transform)-dequantizing section 19, adder 20, generated bitamount extracting section 21, parameter value control section 22, andintra/inter determination section 23.

[0052] Encoding processes performed by this moving image encodingapparatus are roughly classified into processes in the intra-encodingmode of directly encoding each pixel information in an input framewithout motion compensation prediction and processes in the inter-frameencoding mode of encoding difference information between an originalimage and a motion compensation predicted image obtained by motioncompensation prediction.

[0053] An arrangement for processing in the intra-encoding mode will bedescribed first.

[0054] Referring to FIG. 1, in the intra-encoding mode, a switchingcontact S of the mode switching section 11 is connected to a contact Ain the open state. Moving image data (luminance information and colordifference information) obtained by digital processing in an image inputsection (not shown) are supplied to the block generating section 12 inunits of frames. The block generating section 12 divides each frame ofthe input moving image data in units of sets of a predetermined numberof pixels into a plurality of blocks (macroblocks), combines therespective pixels data in units of blocks, and outputs the resultantdata as block data. Each divided block data is supplied to theDCT-quantizing section 14 through the subtracter 13. In this case, inthe intra-encoding mode, since the switching contact S of the modeswitching section 11 is connected to the contact A in the open state,the data input to the subtracter 13 is directly output to theDCT-quantizing section 14.

[0055] The DCT-quantizing section 14 obtains DCT coefficients bydiscrete-cosine-transforming the input block data, and quantizes the DCTcoefficients. The resultant quantized data is supplied to thevariable-length encoding section 15. The variable-length encodingsection 15 variable-length-encodes the quantized DCT coefficient datainput in units of blocks, together with control data such as a blockmode and parameters. This variable-length encoding is processing inwhich a short code is assigned to a signal with a high probability ofoccurrence, and a long code is assigned to a signal with a lowprobability of occurrence, thereby decreasing the average code length.This processing result is output as intra-encoded data.

[0056] An arrangement for processing in the inter-frame encoding modewill be described next.

[0057] Referring to FIG. 1, in the inter-frame encoding mode, theswitching contact S of the mode switching section 11 is connected to acontact B to which motion compensation predicted image data is supplied.Local decoded image data forming a motion compensation predicted imageof the next input frame by the processing to be described late is storedin the frame memory 16 in units of frames. This local decoded image datais supplied to the motion estimation section 17, together with blockdata output from the block generating section 12. The motion estimationsection 17 compares the local decoded image data with the original imageblock data of the input frame output from the block generating section12 to detect the motion vector of the input frame. More specifically,the moving direction and distance of the object are detected byreferring to the values of neighboring pixels one or more framespreceding the current frame. The resultant motion vector data issupplied to the motion compensation section 18 and variable-lengthencoding section 15.

[0058] The motion compensation section 18 generates predicted image datacorresponding to an original image block from a local decoded imagebased on neighboring pixels one or more frames preceding the currentframe by extracting the value of a pixel corresponding to the motionvector detected by the motion estimation section 17 as a predictionvalue. This predicted image data is supplied to the subtracter 13through the mode switching section 11 in units of blocks. The subtracter13 calculates difference image data between the predicted image data andthe original image block data of the input frame input from the blockgenerating section 12. This difference image data is supplied to theDCT-quantizing section 14 to be converted into DCT coefficientquantization data by DCT and quantization. This data is supplied to thevariable-length encoding section 15. This variable-length encodingsection 15 then variable-length-encodes this data together with controldata such as a block mode, parameters, and motion vector and outputs theresultant data as inter-encoded data.

[0059] In the above inter- and inter-frame encoding modes, an outputfrom the DCT-quantizing section 14 is supplied to the IDCT-dequantizingsection 19. The IDCT-dequantizing section 19 performs dequantization andinverse DCT-processing for the output data from the DCT-quantizingsection 14. In intra-encoding, this processing result is supplied aslocal decoded image data to the frame memory 16 through the adder 20 andstored in units of frames. In the inter-frame encoding mode, thedifference image data obtained by the IDCT-dequantizing section 19 issupplied to the adder 20. The adder 20 adds the motion compensationpredicted image data supplied through the mode switching section 11 tothe difference image data and decodes the data. The resultant data isthen stored in the frame memory 16 in units of frames. The image datastored in the frame memory 16 is used for motion compensation predictionof the next frame.

[0060] An arrangement for performing intra/inter switching control onthe mode switching section 11 will be described next.

[0061] The amount of encoded bits generated by the variable-lengthencoding section 15 is monitored by the generated bit amount extractingsection 21 in units of macroblocks. More specifically, the generated bitamount extracting section 21 measures a bit count MB_bits ofvariable-length encoded data from the variable-length encoding section15 in units of macroblocks, and supplies the count result to theparameter value control section 22.

[0062] The parameter value control section 22 compares the encoded databit count MB_bits input in units of macroblocks with a preset referencevalue bits_Thr, and updates a parameter value T(n) (n represents the nthmacroblock output from the block generating section 12) corresponding tothe comparison result. The update result is supplied to an intra/interdetermination section 23.

[0063] The intra/inter determination section 23 compares the inputparameter value T(n) with a preset boundary value refresh_Thr todetermine the intra-encoding mode or inter-frame encoding mode inaccordance with the comparison result, generates a control signal forsetting the corresponding encoding mode, and outputs it to the modeswitching section 11.

[0064] The overall flow of processing for encoding a 1-frame movingimage in the moving image encoding apparatus having the abovearrangement will be described with reference to the flow chart of FIG.2.

[0065] The block generating section 12 divides pixel data constituting1-frame moving image, input from the image input section (not shown),into macroblocks each having 16×16-pixel luminance data and two8×8-pixel color difference data (steps S201 and S202). The respectiveblock data are output from the block generating section 12 in apredetermined order, and it is determined according to the procedureshown in FIG. 3 whether intra-encoding or inter-encoding is performed(step S203). With regard to an initial frame or a frame exhibiting nocorrelation with previous frame, all the block data are processed in theintra-encoding mode, and the parameter value T(n) of each block is resetto 0.

[0066] In step S203, the subtracter 13 calculates the difference between0 and the data of each macroblock for which the intra-encoding mode isselected (step S211). The DCT-quantizing section 14 converts thisdifference data into DCT coefficients in units of 8×8-pixel blocks bydiscrete cosine transform (so-called DCT computation) and quantizes theDCT coefficients (steps S212 and S213). Thereafter, the variable-lengthencoding section 15 variable-length-encodes the DCT coefficientquantization data together with mode information and the like, andoutputs the resultant data as intra-encoded data (step S214).

[0067] The DCT coefficient data quantized by the DCT-quantizing section14 is subjected to dequantization and inverse cosine transform (inverseDCT computation) concurrently with the processing in step S214, and theresultant data is stored as local decoded image data for motioncompensation prediction for the next frame in the frame memory 16 (stepsS208 to S210).

[0068] In step S203, the motion estimation section 17 compares the dataof a macroblock for which the interframe encoding mode is selected withthe local decoded image data stored in the frame memory 16 to calculatea motion vector (step S204). The motion compensation section 18 thenforms a motion compensation predicted image based on local decodedimages from the local decoded image data of temporally different frames,stored in the frame memory 16, and the motion vector calculated by themotion detecting section 17 (step S205). The subtracter 13 obtainsdifference values between the pixel data of the block output from theblock generating section 12 and the pixel data of the predicted image ofthe corresponding block, thereby forming a difference image (step S206).

[0069] The DCT-quantizing section 14 performs DCT processing andquantization for this difference image data in units of 8×8-pixel blocks(steps S212 and S213), thus obtaining quantized DCT coefficient data. Inthe inter-frame encoding mode, the IDCT-dequantizing section 19 performsdequantization and inverse DCT processing for the quantized DCTcoefficient data (steps S207 and S208). The adder 20 adds the motioncompensation predicted image data to the resultant data to form aninternal decoded image (step S209). The resultant decoded image data isstored in the frame memory 16 (step S210). This stored image data isused for motion compensation prediction of the next frame.

[0070] The variable-length encoding section 15 variable-length-encodesthe quantized DCT coefficient data after DCT-quantization in units ofmacroblocks, together with control data such as encoding modeinformation and motion vector, and outputs the resultant data asinter-encoded data (step S214).

[0071] If the macroblock variable-length-encoded in step S214 is not thefinal block of the input frame, the flow returns to step S201 to startencoding processing for the next frame (step S215).

[0072] The intra/inter determination processing in step S203 will bedescribed next with reference to the flow chart of FIG. 3. Note that asymbol (A) in FIG. 3 is connected to “(A)” in FIG. 2, and indicates thatprocessing continuously performed from “(A)” in FIG. 2.

[0073] Assume that the quantized DCT coefficient data of the nthmacroblock is variable-length-encoded by the variable-length encodingsection 15 in step S214 in FIG. 2. In this case, the number of generatedbits of the encoded data is monitored by the generated bit amountextracting section 21. That is, the generated bit amount extractingsection 21 measures the generated bit count MB_bits of the macroblockgenerated by the variable-length encoding section 15 (step S301). Thisgenerated bit count MB_bits is sent to the parameter value controlsection 22.

[0074] The parameter value T(n) handled by the parameter value controlsection 22 is one of the parameters of the nth macroblock. The parametervalue control section 22 compares the generated bit count MB_bits withthe preset reference bit count bits_Thr (step S302). If the generatedbit count MB_bits exceeds the reference value bits_Thr, the parametervalue T(n) is incremented by one (step S303). Assume that the generatedbit count MB_bits obtained by variable-length encoding of the nthmacroblock is 100, and the present reference bit count bits_Thr is 75.In this case, since the generated bit count MB_bits exceeds thereference value bits_Thr, the parameter value T(n) is incremented byone.

[0075] The intra/inter determination section 23 compares the parametervalue T(n) with the preset boundary value refresh_Thr before encoding ofthe nth macroblock is started in the next frame (step S304). IfT(n)>refresh_Thr, a control signal is output to the mode switchingsection 11 to perform encoding in the intra-encoding mode.

[0076] As a consequence, in the mode switching section 11, the switchingcontact S is connected to the contact A so as to set the intra-encodingmode (step S305), and encoding is performed in the intra-encoding modeto execute refresh (step S307). Upon completion of encoding in theintra-encoding mode, the parameter value T(n) is reset to 0 (step S308).

[0077] If T(n)<refresh_Thr in step S304, a control signal is output tothe mode switching section 11 to perform encoding in the inter-frameencoding mode. As a consequence, in the mode switching section 11, theswitching contact S is connected to the contact B to set the inter-frameencoding mode to perform encoding in the inter-frame encoding mode (stepS306).

[0078] In the moving image encoding apparatus of this embodiment, thebit count MB_bits of data variable-length-encoded in units ofmacroblocks is measured, the reference value bits_Thr as a condition forupdating the parameter value T(n) in each block is set for the bit countMB_bits, and the boundary value refresh_Thr of the parameter value T(n)is set in advance as a criterion for the selection of the intra-encodingmode. In the inter-frame encoding mode, the bit count MB_bits of encodeddata is measured in units of blocks, and when the count value MB_bitsexceeds the reference value bits_Thr, the parameter value T(n) of thecorresponding block is updated. A block from which a large amount ofinformation is generated is more likely to receive an error than a blockfrom which a small amount of information is generated. That a statewherein a large amount of information is generated is repeated severaltimes indicates that the data of neighboring blocks are referred to bymotion compensation to a high degree. In this case, if an error ispresent in the referred data, the error is likely to propagatespatially. For this reason, it is determined that the influence of adata error is large in a block in which the parameter value T(n) exceedsthe boundary value refresh_Thr, and the current mode is switched to theintra-encoding mode to perform refresh operation.

[0079] As described above, in the moving image encoding apparatus ofthis embodiment, whether refresh is required can be accuratelydetermined in units of macroblocks in one frame. Even if the techniqueof cyclically performing refresh by intra-encoding is used, refresh fora given macroblock is executed upon switching to the intra-encoding modewhenever refresh is required without waiting for the next refresh cycle.This makes it possible to prevent a deterioration in image quality dueto a data error from remaining and the data error from diffusing.

[0080] (Second Embodiment)

[0081]FIG. 4 is a block diagram showing the arrangement of a movingimage encoding apparatus according to the second embodiment of thepresent invention.

[0082] The moving image encoding apparatus of the second embodiment ischaracterized in that whether to intra-encode macroblocks in the nextframe or inter-encode them is selected by monitoring whether a non-zerocoefficient amount obtained by a DCT-quantizing section 14 is large orsmall in units of macroblocks. If the number of non-zero coefficients ofthe DCT coefficients obtained by DCT computation is large, it can bedetermined that a large variation is occurring in the input image. Byusing this, non-zero coefficients are counted, and it is determined inaccordance with this count value whether to perform refresh operation ornot.

[0083] The moving image encoding apparatus shown in FIG. 4 is comprisedof a mode switching section 11, a block generating section 12, asubtracter 13, the DCT-quantizing section 14, a variable-length encodingsection 15, a frame memory 16, a motion estimation section 17, a motioncompensation section 18, an IDCT-dequantizing section 19, an adder 20, aparameter value control section 22, an intra/inter determination section23, and a total coefficient count extracting section 24. The functionalblocks denoted by the same reference numerals as in FIG. 1 have the samefunctions in FIG. 4.

[0084] The moving image encoding apparatus of the second embodimentdiffers from the apparatus of the first embodiment in that the totalcoefficient count extracting section 24 is used in place of thegenerated bit amount extracting section 21. The total coefficient countextracting section 24 extracts the total number of non-zero coefficientsby counting nonzero coefficients of the DCT coefficients obtained bydiscrete cosine transform from the DCT-quantizing section 14 in units ofmacroblocks.

[0085] In the moving image encoding apparatus having the abovearrangement, the overall processing flow for moving image encoding ofone frame is shown in FIG. 2, and a description thereof will be omitted.A procedure for executing refresh by intra-encoding in step S203 in FIG.2 will be described with reference to the flow chart of FIG. 5. Notethat a symbol (B) in FIG. 5 is connected to “(B)” in FIG. 2, andindicates that processing is continuously performed from “(B)” in FIG.2.

[0086] Assume that the DCT coefficients of the nth macroblock areobtained in step S212 in FIG. 2. In this case, the total coefficientcount extracting section 24 counts only non-zero coefficients of the DCTcoefficients in the macroblock and extracts a total count n_coef (stepS501). This total count n_coef of non-zero coefficients is sent to theparameter value control section 22. The parameter value control section22 compares the total count n_coef of non-zero coefficients in the nthmacroblock with a prescribed value tcoef_Th for non-zero coefficients(step S502).

[0087] If n_coef>tcoef_Th, the value of a parameter T(n) of the nthmacroblock is incremented by one (step S503). The value of thisparameter T(n) is sent to the intra/inter determination section 23.

[0088] The intra/inter determination section 23 compares the parametervalue T(n) with a preset boundary value refresh_Thr before encoding ofthe nth macroblock is started in the next frame (step S504). IfT(n)>refresh_Th4, a control signal is output to the mode switchingsection 11 to perform encoding in the intra-encoding mode. As aconsequence, in the mode switching section 11, a switching contact S isconnected to a contact A to set the intra-encoding mode (step S505).Encoding is then performed in the intra-encoding mode and refreshoperation is executed (step S507). After encoding in the intra-encodingmode, the parameter value T(n) is reset to 0 (step S508).

[0089] If it is determined in step S502 that n_coef>tcoef_Th is notsatisfied, or it is determined in step S504 that T(n)<refresh_Thr issatisfied, a control signal is output to the mode switching section 11so as to perform encoding in the inter-frame encoding mode. As aconsequence, in the mode switching section 11, the switching contact Sis connected to a contact B to set the inter-frame encoding mode, andencoding is performed in the inter-frame encoding mode (step S506).

[0090] In the moving image encoding apparatus of this embodiment, onlynon-zero coefficients in the DCT coefficients in a macroblock arecounted to extract the total count n_coef, and the prescribed valuetcoef_Th as an update condition for the parameter value T(n) in eachblock is set with respect to the total count n_coef of non-zerocoefficients. In addition, the boundary value refresh_Thr of theparameter value T(n) is set in advance as a criterion for the selectionof the intra-encoding mode. In the inter-frame encoding mode, the totalDCT non-zero coefficient count n_coef is obtained for each block, andwhen the total count n_coef exceeds the prescribed value tcoef_Th, theparameter value T(n) of the corresponding block is updated. A block fromwhich a large amount of information is generated is more likely toreceive an error than a block from which a small amount of informationis generated. That a state wherein a large amount of information isgenerated is repeated several times indicates that the data ofneighboring blocks are referred to by motion compensation to a highdegree. In this case, if an error is present in the referred data, theerror is likely to propagate spatially. For this reason, it isdetermined that the influence of a data error is large in a block inwhich the parameter value T(n) exceeds the boundary value refresh_Thr,and the current mode is switched to the intra-encoding mode to performrefresh operation.

[0091] As described above, in the moving image encoding apparatus ofthis embodiment, as in the first embodiment, whether refresh is requiredor not can be properly determined for each macroblock in one frame. Evenif the technique of cyclically performing refresh by intra-encoding isused, refresh for a given macroblock is executed upon switching to theintra-encoding mode whenever refresh is required without waiting for thenext refresh cycle. This makes it possible to prevent a deterioration inimage quality due to a data error from remaining and the data error fromdiffusing.

[0092] In this case, in moving image encoding, the data of eachmacroblock is subjected to discrete cosine transform and quantization,and the resultant data is subjected to variable-length encoding. Ingeneral, for each macroblock, the number of non-zero coefficients isobtained. In the present invention, since non-zero coefficients-in DCTcoefficients subjected to discrete cosine transform are counted in unitsof macroblocks, a non-zero coefficient count can be easily obtainedwithout adding any new functional block.

[0093] (Third Embodiment)

[0094]FIG. 6 is a block diagram showing the arrangement of a movingimage encoding apparatus according to the third embodiment of thepresent invention.

[0095] The moving image encoding apparatus according to the thirdembodiment is characterized in that both or one of an update conditionfor a parameter value T(n) and a criterion for the selection ofintra/inter-encoding is updated in accordance with an error rate in atransmission path.

[0096] Referring to FIG. 6, the moving image encoding apparatus iscomprised of a mode switching section 11, block generating section 12,subtracter 13, DCT-quantizing section 14, variable-length encodingsection 15, frame memory 16, motion estimation section 17, motioncompensation section 18, IDCT-dequantizing section 19, adder 20,parameter value control section 22, intra/inter determination section23, total coefficient count extracting section 24, and error ratedetecting section 25. The functional blocks denoted by the samereference numerals as in FIGS. 1 and 4 have the same functions in FIG.6.

[0097] The moving image encoding apparatus of the third embodiment isbased on the apparatus arrangement of the second embodiment, andadditionally has the error rate detecting section 25. The error ratedetecting section 25 detects the error rate of data errors that haveoccurred in a transmission path (not shown). The detected error rate issent to the parameter value control section 22 and intra/interdetermination section 23 to be used to update an update condition for aparameter value T(n) and a criterion for the selection ofintra/inter-encoding. In the parameter value control section 22 andintra/inter determination section 23, a plurality of reference valuesfor evaluating the error rate of transmission path errors are preparedin advance.

[0098] This embodiment is based on the arrangement of the secondembodiment. However, the same effects can be obtained even if the thirdembodiment is based on the arrangement of the first embodiment.

[0099] In the moving image encoding apparatus having the abovearrangement, the overall flow of processing for encoding a 1-framemoving image is shown in FIG. 2, and a description thereof will beomitted. The procedure for executing refresh by intra-encoding in stepS203 in FIG. 2 will be described with reference to the flow chart ofFIG. 7. Symbols (A) and (B) in FIG. 7 are respectively connected to“(A)” in FIG. 3 and “(B)” in FIG. 5, and indicate that processing isperformed at the respective destinations. That is, the flow from “(A)”is based on the arrangement of the first embodiment, whereas the flowfrom “(B)” is based on the arrangement of the second embodiment.

[0100] First of all, data errors that have occurred in a transmissionpath are detected as an error rate Er by the error rate detectingsection 25, and the result is sent to the parameter value controlsection 22 and intra/inter determination section 23 (step S701). Each ofthe parameter value control section 22 and intra/inter determinationsection 23 compares the input error rate Er with a plurality ofreference values (α1 to αk) stepwise (step S702).

[0101] If the detected error rate Er exceeds one of the reference valuesα1 to αk, both or one of a reference total zero coefficient count valuetcoef_Th (a reference bit count value bits_Thr when this embodiment isbased on the arrangement of the first embodiment) preset as an updatecondition for the parameter value section 19 and a boundary valuerefresh_Thr preset as a criterion for the intra/inter determinationsection 23 is decreased (step S703).

[0102] Subsequently, the processing described above is performed at thedestination indicated by “(B)” in FIG. 5 (or “(A)” in FIG. 3) by usingthe corrected reference total coefficient count value tcoef_Th (orreference bit count value bits_Thr) and boundary value refresh_Thr.

[0103] As described above, in the moving image encoding apparatusaccording to this embodiment, data errors in the transmission path aremonitored, and a plurality of reference values of error rates in thetransmission path are prepared. If the detected error rate exceeds oneof the reference values, both or one of the update condition value for aparameter value and the criterion value for the selection ofintra-encoding is decreased. With this operation, the problem that adeterioration in image quality remains on the moving image decodingapparatus side can be solved by increasing the number of macroblocks tobe intra-encoded while finely performing refresh operation in theintra-encoding mode in units of macroblocks.

[0104] (Fourth Embodiment)

[0105]FIG. 8 is a block diagram showing the arrangement of a movingimage encoding apparatus according to the fourth embodiment of thepresent invention.

[0106] The fourth embodiment is characterized in that in the arrangementof the second embodiment, since a transmission rate control buffer isgenerally provided, both or one of the update condition for a parametervalue T(n) and the criterion for the selection of intra/inter-encodingis changed in accordance with the encoded data stored in this ratecontrol buffer.

[0107] Referring to FIG. 8, the moving image encoding apparatus iscomprised of a mode switching section 11, block generating section 12,subtracter 13, DCT-quantizing section 14, variable-length encodingsection 15, frame memory 16, motion estimation section 17, motioncompensation section 18, IDCT-dequantizing section 19, adder 20,parameter value control section 22, intra/inter determination section23, total coefficient count extracting section 24, rate control buffer26, and buffer amount measuring section 27. The functional blocksdenoted by the same reference numerals as in FIGS. 1 and 4 have the samefunctions in FIG. 8.

[0108] The moving image encoding apparatus according to the fourthembodiment is based on the apparatus arrangement of the secondembodiment, and additionally has the rate control buffer 26 and bufferamount measuring section 27.

[0109] The rate control buffer 26 has a buffer memory for multiplexingvariable-length-encoded image data with other data such as audio dataand storing the multiplexed data. This amount of data stored in thisbuffer memory is controlled within a predetermined range to keep theencoding rate constant.

[0110] The buffer amount measuring section 27 counts the amount ofencoded data stored in the rate control buffer 26 in units of frames,and compares the count value with a preset reference value. If the countvalue is equal to or smaller than the reference value, the result issent to the parameter value control section 22 and intra/interdetermination section 23.

[0111] This embodiment is based on the arrangement of the secondembodiment. However, the same effects can be obtained even if the fourthembodiment is based on the arrangement of the first embodiment.

[0112] In the moving image encoding apparatus having the abovearrangement, the overall flow of processing for encoding a 1-framemoving image is shown in FIG. 2, and a description thereof will beomitted. The procedure for executing refresh by intra-encoding in stepS203 in FIG. 2 will be described with reference to the flow chart ofFIG. 9. Note that a symbol (C) in FIG. 9 is connected to “(C)” in FIG.2, and symbols (A) and (B) are respectively connected to “(A)” in FIG. 3and “(B)” in FIG. 5, and indicate that processing is performed at therespective destinations. The flow indicated by “(A)” is based on thearrangement of the first embodiment, whereas the flow indicated by “(B)”is based on the arrangement of the second embodiment.

[0113] The encoded DCT coefficient data of each block obtained in stepS214 in FIG. 2 is stored in the buffer memory of the rate control buffer26 (step S901). The buffer memory sends out the encoded data in units offrames. The buffer amount measuring section 27 measures an encoded dataamount buf_cod stored in this buffer memory (step S902). The obtainedmeasurement value is compared with a preset reference value buf_Thr(step S903).

[0114] If buf_Thr>buf_cod, a request signal is output to both or one ofthe parameter value control section 22 and intra/inter determinationsection 23 to change both or one of the update condition for a parametervalue T(n) and the criterion for the selection of intra/inter-encoding.As a result, both or one of a reference total coefficient count valuetcoef_Th (a reference bit count value bits_Thr when the fourthembodiment is based on the arrangement of the first embodiment), whichis an update condition for the parameter value control section 22, and aboundary value refresh_Thr, which is a criterion for the intra/interdetermination section 23, is decreased (step S904).

[0115] Subsequently, the above processing at the destination (B) in FIG.5 (or (A) in FIG. 3) is performed by using the corrected reference totalcoefficient count value tcoef_Th (or reference bit count value bits_Thr)and boundary value refresh_Thr.

[0116] As described above, in the moving image encoding apparatus ofthis embodiment, the buffered data amount in the rate control buffer 26is measured, and when the measured buffered data amount is equal to orsmaller than a prescribed value, both or one of the update conditionvalue for a parameter value and the criterion value for setting ofintra-encoding is decreased. With this operation, the problem that adeterioration in image quality remains on the moving image decodingapparatus side can be efficiently solved by increasing the number ofmacroblocks to be intra-encoded while finely performing refreshoperation in the intra-encoding mode in units of macroblocks.

[0117] In general, when the buffered data amount in the rate controlbuffer 26 becomes equal to or smaller than the prescribed value, dummybits (so-called stuffing bits), which are discarded on the moving imagedecoding apparatus side, are added to output data to keep thetransmission rate constant. In the fourth embodiment, the number ofmacroblocks subjected to refresh by intra-encoding is increased toincrease the amount of encoded data stored in the rate control buffer.This makes it possible to decrease the number of dummy bits and performrefresh control efficiently using the amount of codes encoded when theamount of codes encoded is small.

[0118] The present invention is not limited to the above embodiments.Various changes and modifications of the embodiments can be made withinthe spirit and scope of the invention. For example, the encoding schemebased on motion compensation prediction has been exemplified as aprediction encoding scheme. However, the same effects as those describedabove can be obtained by using encoding techniques such as backgroundprediction and inter-frame difference suppression.

[0119] As has been described above, according to the present invention,a parameter value update condition for each macroblock and a criterionfor the selection of an encoding mode are set, and each parameter valueis updated in accordance with the number of bits variable-length-encodedin units of macroblocks and the count value of non-zero coefficients inDCT coefficients subjected to discrete cosine transform. When thisparameter value exceeds a present value, refresh operation based onintra-encoding encoding is executed. This makes it possible toaccurately determine whether to require refresh in encoding eachmacroblock and suppress a deterioration in image quality due to a dataerror and diffusion of the data error.

[0120] When the error rate in the transmission path exceeds apredetermined value or the buffered data amount in the rate controlbuffer becomes equal to or less than a prescribed value, both or one ofthe update condition value for a parameter value and the criterion valuefor the selection of intra-encoding is decreased. Therefore, the problemthat a deterioration in image quality remains on the moving imagedecoding apparatus side can be solved by efficiently increasing thenumber of macroblocks to be intra-encoded.

[0121] Furthermore, when the buffered data amount in the rate controlbuffer is defined, the buffered data amount is monitored, and an updatecondition and criterion are properly set in accordance with the buffereddata amount. With this operation, even if the amount of codes to beencoded is small, refresh operation based on intra-encoding is properlyexecuted, thus preventing the propagation of a data error.

1. A moving image encoding apparatus including block generating means(12) for receiving moving image data in units of frames, dividing thereceived data into a plurality of blocks, and outputting the data of therespective blocks in a predetermined order, and mode-specific encodingmeans (11, 13-20), having intra-encoding mode and inter-frame encodingmode, for encoding in a designated mode the data of the respectiveblocks sequentially output from said block generating means,characterized by comprising: information extraction means (21, 24) forextracting information for determination of a degree of influence of adata error from an encoded block processed by said mode-specificencoding means (11, 13-20); parameter value control means (22) for, whenintra-encoding is performed by said mode-specific encoding means (11,13-20), setting an initial value as a parameter value of a correspondingblock in advance, comparing the information extracted by saidinformation extraction means (21, 24) with a predetermined referencevalue, and updating and storing the parameter value on the basis of thecomparison result; and intra/inter determination means (23) for, whendata of a block of a next frame, output from said block generating means(11), is to be encoded, comparing a parameter value of a correspondingblock, stored in said parameter value control means (22), with apredetermined boundary value, determining on the basis of the comparisonresult whether to execute refresh by intra-encoding, and designating amode based on the determination result with respect to saidmode-specific encoding means (11, 13-20).
 2. An apparatus according toclaim 1, characterized in that said mode-specific encoding means (11,13-20) encodes pixel components in an input frame in units of blockswithout performing motion compensation prediction in the intra-encodingmode, and obtaining difference values between an original image and amotion compensation predicted image obtained by motion compensationprediction and encoding the difference values in units of blocks in theinter-frame encoding mode.
 3. An apparatus according to claim 1,characterized in that said information extraction means (21) extracts agenerated encoded data amount from an encoded output from saidmode-specific encoding means as information for determination of thedegree of influence of the data error.
 4. An apparatus according toclaim 1, characterized in that when said mode-specific encoding means(11, 13-20) performs discrete cosine transform processing for blockdata, said information extraction means (24) extracts a total number ofnon-zero coefficients of coefficients obtained by the discrete cosinetransform processing as information for determination of the degree ofinfluence of the data error.
 5. An apparatus according to claim 1,characterized by further comprising reference value/boundary valuechanging means (25) for detecting an error rate in a transmission path,and changing at least one of a reference value held in said parametervalue control means and a boundary value held in said intra/interdetermination means (23) on the basis of the detected error rate.
 6. Anapparatus according to claim 1, characterized by further comprising:encoded data storage means (26) for storing encoded data output fromsaid mode-specific encoding means (11, 13-20) in units of blocks; andreference value/boundary value changing means (27) for obtaining anencoded data amount stored in said means (26), and changing at least oneof a reference value held in said parameter value control means and aboundary value held in said intra/inter determination means (23) on thebasis of the encoded data amount.
 7. An apparatus according to claim 1,characterized in that said parameter value control means (22) adds oneto the parameter value when information extracted by said informationextraction means is larger than a reference value, and adds 0 to theparameter value otherwise.